Goto

Collaborating Authors

 predictive estimability


Double Descent: Understanding Linear Model Estimation of Nonidentifiable Parameters and a Model for Overfitting

Christensen, Ronald

arXiv.org Machine Learning

We consider ordinary least squares estimation and variations on least squares estimation such as penalized (regularized) least squares and spectral shrinkage estimates for problems with p > n and associated problems with prediction of new observations. After the introduction of Section 1, Section 2 examines a number of commonly used estimators for p > n. Section 3 introduces prediction with p > n. Section 4 introduces notational changes to facilitate discussion of overfitting and Section 5 illustrates the phenomenon of double descent. We conclude with some final comments.